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ment change monitoring agent. For a document recommen- 
dation system to be effective, it is desirable to enable users 
to be cognizant of changes that may occur to the document. 
The present invention addresses this issue by coupling a 
document change monitoring agent, which automatically 
detects changes in referenced documents, with a recom- 
mender system, which helps users share and evaluate infor- 
mation in a collaborative way. One advantage of the inven- 
tion is that it brings human judgement into the relevance 
evaluation of the detected changes and allows the results to 
be shared with other people Hkely to be interested, in such 
a way that redundant work is decreased. Another advantage 
of the invention is that it enables Systems Administrators of 
a document recommendation system to be more eflBcient in 
the management of the system. 
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SYSTEM FOR PROVIDING DOCUMENT Changes to documents are also of interest to users. Such 

CHANGE INFORMATION FOR A changes can take many forms: substantive content change, 

COMMUNITY OF USERS cosmetic/syntactic changes, and disappearance of the docu- 
ment. In addition, administrators of data stores containing 
UTCT TTJT? TNn/T^Km^KT ^ document references face the additional problem that the 

MhLU Vt iHb INVENTION ^^me document can be referenced by multiple references. 

The present invention is related to the field of information Some repositories and databases are equipped with tools, 

retrieval, and in particular to the field of providing document using triggers, that help users deal with these problems by 

change information to a user, notifying users when referenced items of interest change in 

10 some way or are removed. On the WWW, there are now also 

BACKGROUND OF THE INVENTION several services available to help users monitor Web pages 

The ever-increasing universe of electronic information, ^^^^ ^^^^^ Unifonm Resource Locator (URL) address, 

for example as found on the World Wide Web (herein after such as: netmind, Smart Bookmarks, Grassroots, 

referred to as the Web), competes for the effectively fixed "WebTracker— a Web Service for tracking documents" by 

and limited attention of people. Both consumers and pro- Fishkin, K. and Bier, E. 
ducers of information want to understand what kinds of 

information are available, how desirable it is, and how its netmind: http://www.netmind.com, 

content and use change through time. Smart Bookmarks: http://www.firstfloor.com/ 

Making sense of very large collections of linked docu- ^ SmartBookmarks2.0/QuickStart.htmI, 
ments and foraging for information in such environments is '° ^^fu^""^' http://wvAjw-^^^^^^ 

difBcult without specialized aids. Collections of linked WebTracker— a Web Service for tracking documents," by 

documents are often connected together using hypertext Fishkin, K. and Bier, E. available at http:// 

links. The basic structure of Unked hypertext is designed to www.parc.xerox.com/istl/members/fishkin/doc/ 

promote the process of browsing from one document to webtracker.html. 

another along hypertext links, which is unfortunately very - i- ^ 

slow and inefficient when hypertext collections become very ^^^^^^^ ^^^s they have regis- 

large and heterogeneous. Two sorts of aids have evolved in ^^'^"^ ^'^^ ^^^^ ^^""^ changed m some way. In 

such simations. The first are structures or tools that abstract addition, they may also be able to federate the change 

and cluster information in some form of classification sys- momtonng systems of other kinds of networked reposito- 

tem. Examples of such would be library card catalogs and However, evaluating the nature of the change and its 

the Yahoo! Web site (URL http://www.yahoo.com). The iiriportance for the user is a difficult task to automate and is 

second are systems that attempt to predict the information ^^""^ weakness of such change monitoring systems, pe 

relevant to a user's needs and to order the presentation of notifications while perhaps including a gr^at deal of 
information accordingly. Examples would include search „ data concerning the change, may be potentially irrelevant to 

engines such as Lycos (URL: http://www.lycos.com), which ^'^^^ ^^^io 

take a user's specifications of an information need, in the "^"^ ^'^^ annoyance than aid. The most 

form of words and phrases, and return ranked lists of ^^^^^^^^ to evaluate the nature and importance the 

documents that are predicted to be relevant to the user's ^**^"Se remams the user him/herself, at least for the fore- 



need. 



seeable future. 



A ^. 1 r , , . J „ 40 In addition, within a given workgroup, work community, 

Another class or tools is recommender systems. Recom- . • i-i i *l * *u ■ . r u . 

J , - J 1 • . r J 1 . or organization, it is likely that the existence of substantive 

mender systems provide a list of recommended subsequent . • ^ ^ nu i . r 

. ^ • • L 1 J • J changes in a document will be relevant to a number of 

web pages worth viewing based on some predetermined i , • . i *u- *u ^ c i 

(iu ■ : ' u J . V • «r. people, not just one. In this case, the work of evaluating the 

filtering cri ten a. One such recommender tool IS the Rec- * u ■ ii i * u j * u . 

°,„ f , . J , Ai T * . «7 u nature of the change is likely to be done not once but many 

ommend feature provided on the Alexa Internet Web site ac *u • *i j * u t 

nmr i,** //. i \ j» ^ * times, as there IS currently no good way to share this work. 

(URL: http://www.alexa.com). The "Recommend feature / & j 

provides a list of related Web pages that a user may want to SUMMARY OF THE INVENTION 

retrieve and view based on the Web page that they are a document recommendation system incorporating a 

currently viewing. document change monitoring agent is disclosed. As 

Another recommender system is termed "the Knowledge 50 described above with respect to the prior art, there are a 

Pump" and is described by Glance, N., Arregui, D. and number of problems that are experienced by users of net- 

Dardenne, M. "Knowledge Pump: Supporting the Flow and worked repositories. The present invention addresses several 

Use of Knowledge." In Information Technology for Knowl- that stem from the fact that access to these repositories is 

edge Management. Eds. U. Borghoff and R. Pareschi, New typically done via document references that point to docu- 

York: Springer- Verlag, pp. 35^5, 1998. The "Knowledge 55 ments. The problems that arise are: (1) the document refer- 

Pump" was designed for use within organizations and has a ence itself may not remain stable (may become stale); (2) the 

key focus on the sharing of information in the form of content of document referenced may change; and (3) mul- 

documents. tiple references may exist for the same item. Solutions that 

Prospective applications of recommender systems are as exist today, such as document change monitoring agents on 

information sharing applications for organizations. Here, the 60 the World Wide Web and change trigger agents within 

quality of the recommendation service is key. If the recom- repositories and databases, are entirely automated processes, 

mender system fails to recognize stale references and and as such, do not resolve these problems satisfactorily. In 

changes in recommended documents, the users' experience addition, they are oriented to single users as opposed to 

of the information sharing environment will suffer signifi- groups of users. 

cantly as a result. In other words, a main failing of recom- 65 The present invention addresses these issues by coupling 

mender systems currently is that they provide static recom- a document change monitoring agent, which automatically 

mendations of potentially dynamic objects. detects changes in referenced documents, with a recom- 
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mender system, which helps users share and evaluate infor- 
mation in a collaborative way. An important advantage of 
the present invention is that it brings human judgement into 
the relevance evaluation of the detected changes and allows 
the results to be shared with other people likely to be 5 
interested, in such a way that redundant work is decreased. 

The present invention shows how this limitation can be 
overcome by, first of all, coupling a document change 
monitoring agent with the recommender system and, 
secondly, implementing a collaborative process for evaluat- 
ing the nature and importance of these changes. It has been 
determined through research and experience with existing 
recommender systems, that such functionality is key to 
improving the usability of a recommender system intended 
for sharing information in the form of documents. is 

A system implementation of the present invention may 
include the following elements: a document recommenda- 
tion element for providing document recommendations to a 
user based on a user profile, said document recommendation 
element further comprising one or more action elements for 
responding to a change in a recommended document, each 
of said one or more action elements performing a different 
action based on the nature of said change; document repre- 
sentation storage, for storing representations of said recom- 
mended documents; a document registration element 
coupled to said document recommendation element, said 
document registration element for enabling a user to identify 
recommended documents for which they would like to be 
notified of changes; a document change monitoring element 
coupled to said document registration element, said docu- 
ment change monitoring element for detecting a change to 
said document and reporting said change to said document 
recommendation element; and a user notification element 
coupled to said document recommendation element, said 
user notification element for notifying users of said change. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram of a change monitoring agent 
as is currently known in the prior art 40 

FIG. 2 a block diagram of a recommender system as is 
currently known in the prior art. 

no. 3 is a flowchart illustrating the steps for coupling a 
change monitoring agent with a recommender system as 
may be performed in the currently preferred embodiment of 
the present invention. 

FIG. 4 is a block diagram further illustrating the interac- 
tions between the change monitoring agent, the recom- 
mender system, the users and the document repositories as 
may be performed in the currently preferred embodiment of 
the present invention. 

FIG. 5 is a block diagram of a computer system which 
may be utilized to implement the system of the currently 
preferred embodiment of the present invention. 55 

DETAILED DESCRIPTION OF THE 
INVENTION 

A system for providing document change information to 
a community of users having an interest in the document is 60 
disclosed. The system provides for collaborative mechanism 
for user to share feedback about such changes. The currently 
preferred embodiment of the present invention is imple- 
mented for analyzing collections of linked documents (i.e. 
Web Pages) residing on the portion of the Internet known as 65 
the World Wide Web (hereinafter the Web). ^Ilie Web is the 
portion of the Internet that is used to store and access linked 
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documents. A Web page may have multi-media content as 
well as relative and absolute links to other pages. A Web 
page is accessed by its Uniform Address Locator (URL) 
address. A Web Site is a collection of related Web pages 
associated with an entity having a presence on the World 
Wide Web such as a company, educational institute or the 
like. A repository (or source/document repository) is a Web 
location containing collections of documents which may be 
accessed via the Web. 

A Web Browser is a tool which enables a user to traverse 
through and view documents residing on the Web. Other 
rendering means associated with the Browser will permit 
listening to audio portions of a document or viewing video 
or image portions of a document. Examples of commercially 
available Web Browsers include Netscape Navigator®, 
available from Netscape Communications Corporation, and 
Internet Explorer, available from the Microsoft® Corpora- 
lion. 

Overview of the Present Invention 

In the present invention a document change monitoring 
agent is coupled with a recommender system for three main 
purposes: (1) to more effectively evaluate the nature and 
importance of changes in referenced documents; (2) to share 
such results among a community of users; and (3) to help 
administrators better manage data stores containing docu- 
ment references (e.g., the data store associated with a 
recommender system). 

The document change monitoring agent, as described in 
detail below, is responsible for tracking changes in refer- 
enced documents. Its main components, with respect to this 
invention, are data storage for storing representations of the 
document content and a function for evaluating the extent 
and character of the change. Such document change moni- 
toring agents exist already on the Web (as described in the 
prior art) and have also been instantiated as triggers within 
databases. Added to the document change monitoring agent 
is means to share information with the second module, the 
recommender system. 

A recommender system is a system that enables users to 
share recommendations in a way that limits information 
overload. An important component of the recommender 
system is a filtering mechanism that, for each user, ranks or 
rates the priority of recommended items coming from oth- 
ers. In the present invention, we are concerned only with 
recommender systems that handle recommendations of 
pointers to items (as opposed to the items themselves). 

By coupling the document change monitoring agent with 
a recommender system, users who have recommended items 
which have since changed can be notified of that change, if 
they so desire. The recommender system is extended to 
provide users with the means to evaluate the nature and 
importance of the change. This evaluation can then be 
shared with other users of the recommender system, using 
the filtering mechanisms that already exist. 

Thus, the present invention, on the one hand, augments 
the automatic process of monitoring document changes with 
human evaluation, and on the other hand, limits the amount 
of redundant evaluation work that goes on by sharing the 
residts of the evaluation with those other people most likely 
to be interested. 

Finally, a number of side benefits ensue from the per- 
spective of administrators of the recommender system (or 
other data store connected to the combined system): (1) the 
administrator is notified when pointers become stale; (2) the 
administrator is notified when two or more references point 
to the same document; (3) the administrator is notified when 
the reference itself has changed; and (3) the administrator is 
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notified when it is likely that access to a particular repository 
is no longer possible, i.e., when that repository has been 
disconnected from the network. 

Described in more detail below are both document change 
monitoring agents and recommender systems as they exist 
today, and additionally a characterization of the nature of the 
source repositories alongside which these operate. Then 
discussed in greater depth is the invention and manner for 
coupling of the change monitoring agent and the recom- 
mender system in order to resolve the problems discussed 
above. Finally, discussed are the changes required in both 
the document change monitoring agent and in a recom- 
mender system in order to implement the present along with 
explicit specifications for both. 
Document Change Monitoring Agent 

FIG. 1 illustrates a document change monitoring agent as 
known in the prior art. A document change monitoring agent 
provides a means to track changes in online documents and 
to notify interested people about the change. Document 
monitoring involves three steps which are illustrated with 
reference to FIG. 1. 

Step 1 User Document Registration and Interest Specifi- 
cation: The user 101 registers with the monitoring 
agent 102 the identifier of the online document 103 (for 
agents operating on the Web, this is typically the URL 
of the document). Additionally some agents permit the 
definition of: i) a region to monitor in the document, ii) 
a Ust of keywords that represent the user interest, iii) 
the periodicity of the check the agent will perform. 
Step 2 Monitoring Agent Monitors Document(s): The 
monitoring agent 102 monitors document changes by 
periodically accessing the document or, alternatively, 
by requiring a notification service from the document 
repository. The latter case is less common, since i) 
document repositories often do not offer this service, ii) 
the added value of the agent resides then mainly in the 
federation of several repositories for monitoring 
changes. In the former case, the monitoring agent 102 
is responsible for detecting the change and therefore 
associates with the document identifier a record of 
information representing the state of the document. 
Such information is stored in document content repre- 
sentation storage 104. The monitoring agent 102 
detects changes by comparing successive records. The 
information recorded may range from the whole docu- 
ment content to a document content checksum of only 
a few bytes. The relevance and efficiency of the change 
detection derives directly from the sort of record(s) (or 
combination thereof) that is used. 
Step 3 Monitoring Agent Notification To User: The moni- 
toring agent 102 notifies the user 101 about the change, 
typically by sending an email describing the detected 
change. Here again, the description of the change can 
range from a detailed textual comparison to an almost 
empty message notifying the occurrence of the change, 
depending on which kind of records were kept for the 
document. 
Recommender Systems 

FIG. 2 is a block diagram illustrating a recommender 
system as is known in the prior art. A recommender system 
uses filtering techniques to provide personalized recommen- 
dations of documents to individual users. Referring to FIG. 
2, the minimal set of features common to any recommender 
system is: 

Data storage 201, which supports at least the following 
three data structures: 

User: personal and profiling information for a member 
of the system. 
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Document: reference to a document (typically a URL). 
Review: associates a user, a document and a numerical 
score (rating). 

A filtering mechanism 202 which ranks or rates recom- 

5 mended items for any given user, based on the user's profile 
(which can contain the user's history of interaction with the 
system, keywords). Common filtering approaches include 
automated collaborative filtering and personalized content 
fihering, which are described in greater detail in the follow- 

10 ing references: Resnick et al. "Group Lens: An Open Archi- 
tecture for Collaborative Filtering of Netnews." Proceedings 
of the ACM 1994 Conference on Computer Supported Coop- 
erative Work, Chapel Hill, N.C., 199', pp. 175-186; and 
Shardanand et al. Social Information Filtering: Algorithms 

15 for Automating Word of Mouth. Proceedings of the ACM 
1995 Conference on Computer Human Interactions y Denver, 
Colo., 1995. pp. 210-217. 

A User interface 203 providing an interface for displaying 
recommended items to a user based on predicted prefer- 

20 ences; an interface for the user to provide feedback on 
recommended items; an interface for allowing the user to 
recommend new items. It should be noted that design of 
such user interfaces are well known in the art, and thus could 
be implemented using such known technology. Thus, no 

25 fiirther discussion on the creation of the user interfaces is 
deemed necessary. 
Source Repositories 

Source repositories are defined here as any repository that 
can serve as a source of recommendations (of documents). 

30 One principle example is the Web, in which every document 
is referenced by a URL. Other examples include networked 
databases and document management systems. Accessing a 
source repository via a reference and the appropriate com- 
munication channel can result in three different results: 

35 1. The document is accessed without problems; 

2. The reference is no longer valid (for example, a URL 
that points to a page that no longer exists); 

3. llie source repository is not responding (for example, 
the HTTP server upon which the source repository 
resides and may be down or taken off the network). 

Couphng a Document Change Monitoring Agent with a 
Recommender System 

A method to couple a document change monitoring agent 
and recommender systems that solves the problems 
described above is described with reference to the flowchart 
of FIG. 3. First, the recommender system registers the 
recommended document to the monitoring agent, step 301. 
This will in turn will monitor changes occurring to the 
document. 

The monitoring agent reports to the recommender system 
any change (change in content or locatibn as well as disap- 
pearance of document) occurring to the document, step 302. 
The monitoring agent also detects multiple registered docu- 
ments having the same content and reports about these 
duplicates. 

Responsive to these reports, the recommender system 
reacts and takes action to the reports as followed: 

Step 303, for relocated documents, it stores the new 
60 location of the document, registers that new location to 
the agent and unregisters the previous one. 
Step 304, for documents whose content has changed, it 
notifies usens implicated by the change (i.e. users 
•having shown interest for the recommended document 
65 and the document submitter, if any). ITiese users are 
given the possibility to annotate again the document or 
to comment on the change. Each user's evaluation of 
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the change is made available to the others, as well as to 
the membership of the recommender system when they 
subsequently receive/access/review the recommenda- 
tion. 

Step 305, for unavailable documents, it shows on the ^ 
display that the document is no longer available and 
possibly notifies the system administrator or the user 
who recommended the document. 
Step 306, for duplicate references to the same document, 
the recommender system updates its representation of 
the document to track the multiple references. 
Based on user requests responsive to their review of 
community feedback relating to the change, the recom- 
mender system might unregister a document from the is 
change monitoring agent, step 307. 

FIG. 4 further illustrates the interactions between the 
agent, the recommender system, the users and the document 
repositories. Referring to FIG. 4, the recommender system 
401 will provide recommended documents to users 402. To 
perform this function, the recommender system 401 inter- 
acts with the document repositories 404. The document 
repositories 404 will typically reside on the Web. In any 
event, the users 402 may then elect to be informed about 
changes to the recommended documents. When this occurs, 
recommender system 401 registers the recommended docu- 
ments with monitoring agent 403. Upon detecting a docu- 
ment change, the monitoring agent 403 reports back to the 
recommender system 401 information about the change. The 
recommender system 401 then notifies the users 402 of the 
change. The user may then provide feedback related to the 
change. 

The method of the present invention solves the problems 
cited above in the following ways: 

Dynamic documents (Le. subject to changes) are handled 
gracefully; users are given the possibility to review 
collaboratively the detected changes and to modify 
their recommendation accordingly. 

Importance of changes is collaboratively evaluated in the 40 
frame of the recommender system. 

Multiple persons no longer need to same efifortAvork to 
assess the value of the change, and in particular, 
insignificant changes need only be identified once. 

Broken links (open or heterogeneous systems) are 45 
detected by the agent, reported to the recommender 
system, which in turn is given the possibility to react 
appropriately. 

Multiple entries having the same content are detected by 
the agent, reported to the recommender system, which ^° 
in turn is given the possibility to react appropriately. 
Monitoring Agent: Specification 

Provided herein is an example, a specification of a moni- 
toring agent (which does not include either the management 55 
of broken links, of relocated documents, or of multiple 
entries). 

The agent as described below runs independently from the 
recommender system. At regular interval it invokes its own 
DoMoniloring( ) method. It should provide some data stor- 60 
age capabilities (however this job could be taken over by the 
recommender system) and should export and Application 
Programming Interface (API). The basic persistent data 
structure at this level is the Monitor. One can understand 
coupling as accessing and manipulating monitors through 65 
the API both by the monitoring agent and the recommender 
system. 
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Monitor: 
Document 

Reference to a document's content. 
LastModified 

Last date and time the document has been found 
modified. Initially Unknown. 
Status 

Ok if the document's content has been accessed with- 
out problems, Error otherwise. 
Initially Ok. 
Characteristic 

Value computed using the document's content, see the 
private methods of the agent provided below. Ini- 
tially its value equals to None. 
Subscribers 

List of users interested in being aware of the changes 
for this particular document. Initially Unknown, 
API Methods: 

For Managing the Monitors: 
Monitor(Document, User) 

Creates a new monitor for a document with a first 
subscriber. 
GetMonitors( ) 

Returns all stored monitors. 
GetMonitor(Document) 

Returns the monitor for a document or None if there is 
not such a monitor. 
AddMonitor(Monitor) 
Adds a monitor. 

RemoveMonitor(Monitor) 

Removes a monitor. 
UpdateMonitor(Monitor) 

Updates the attributes of an existing monitor. 
For Executing the Monitoring Process: 



DoMonitoring( ): 

mons - GetMonitors( ) 
for m in mons: 

d - m.Document 

(sc, c) - FetchContcnt(d) 

if (sc - Ok): 

pc Parse(c) 

m. Status e Ok 

if m. Characteristic =• None: 

m.Characteristic «» Characterize(pc) 

else: 

old = m. Characteristic 
new = Characterize(pc) 
if (not Compare(old, new)): 

m. LastModified » Now( ) 
m.Characteristic - new 

else: 

m. LastModified - Now( ) 
m. Status = Error 
Update Monitor(m) 

Private Methods for Monitor Agent: 
FetchContent(Document) 

Retrieves a document's content (using its URL for 
example) from a source repository, returns a tuple 
(SlatusCode, Content). The result is reported via a Status- 
Code returned by the FetchDocument private method of the 
monitoring agent, which can have three possible values: 

Ok: Document accessed without problems. 

DeprecatedReference: Reference is no longer valid (for 
example URL pointing to a page that doesn't exist any 
longer). 
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AccessFailure: Major problem, the source repository is 
not responding (an HTTP server may be down, for 
example). 
Now() 

Returns the current date and time. 
Parse(Content) 

Parses the content. For example, removing all punctuation 
and HTML mark-ups. Returns parsed-content. 
Characterize(Parsed- Content) 

Returns a value that will remain the same if the docu- 
ment's content remains unchanged. For example, the 
returned value might be generated by a hashcode function. 
Compare(Characteristic, Characteristic) 

Compares two characteristics produced by the Character- 
ize method and returns True if they are considered to be 
similar enough, False otherwise. A threshold value will 
probably be involved in the implementation. 
Recommender System: Minimal Additional Requirements 

The recommender system of the currently preferred 
embodiment builds upon existing systems. Such systems 
would provide the capabilities described above (e.g. a fil- 
tering mechanism and various user interfaces for enabling 
the user to interact with the system). To couple the recom- 
mender system with the specified monitoring agent requires 
at least: 

(1) A way for the recommender system to subscribe 
references to be monitored to the monitoring agent; see 
the Subscribe(Documenl d, User u) method below, 

(2) A way to periodically ask for, or to be notified of, the 
list of changes the monitoring agent has detected; see 
the CheckDocuments( ) method below. 

(3) A way to allow users of the recommender system to 
turn monitoring on and off for any particular reference; 
e.g., a unSubscribe(Document d. User u) method. 

(4) A way to notify the user that the content has changed 
(after suitable filtering) and propose one of several 
actions, e.g., evaluate (review) the change, ignore, stop 
monitoring, advise deletion of reference from data store 
of recommender system; see the Notify Use r(Monitor 
m. User s) method below. 

(5) A way to notify the administrator when the reference 
is unavailable over a long period of time (one week, 
say); see the NotifyAdmin(Monitor m) below. 

Specifically, the recommender system will at regular 
intervals invokes its own CheckDocuments( ) method. The 
time of last invocation is stored in the LastCheck attribute 
and a reference to the monitoring agent is kept in the 
MonitoringAgent attribute. 

In the reviewing interface one additional option is 
provided. Subscribe, which turns monitoring on (or off) for 
the particular reference. 



Subscribe (Document d, User u): 

m » MonitoringAgcnt.GctMonitor(d) 
if (m " None): 

m a new Men iter (d,u) 

Moni toringAgc nt. U pdateMonitor(m) 

Else: 

m.Subscribers.add(u) 
MomtormgAgeDt.UpdatcMonitor(in) 
MomtoringAgeiit.AddMonitor(m) 
CheckDocuments( ): 

mens " MonitoriQgAgem.GetMonitors( ) 
for m in mens: ' 

if (m.LastModified > LastCheck): 
if m.Status - Ok: 
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-continued 

for s in m. Subscribers: 
NotifyUser(m, s) 

5 else: 

No tify Admin (m) 
LastCheck = Now( ) 
NotifyUser(Momtor m. User s): 

Notifies the user s that the content of the reference m. Document 
has changed and proposes several options: 
10 Review 

Fill review about changes that will be accessible to 
all the other users. 
Ignore 

Do nothing, discard the notification. 
Unsubscribe 
35 m. Subscribers. remove(s) 

if (m. Subscribers - [ ]): 

Monito ringAgent. Remove Monito r(m) 

else 

Monito ringAgent. UpdatcMonitor(m) 
NotifyAdmin(Monitor m): 
2Q Notifies the administrator of the recommender system that the 

document corresponding to the reference m. Document is not accessible. 



It should be noted that the monitoring agent and recom- 
mender system may execute on either the same or different 

25 computer systems. Further, a since communication is pref- 
erably through a predetermined Application Programming 
Interface, multiple such monitor agents may operate with a 
single recommender system, or multiple recommender sys- 
tem may operate with a single monitor agent. Implementa- 
tion of such a configuration would not depart from the spirit 

•'^ and scope of the present invention. 

Overview of a Computer Controlled Display System in. the 
Currently Preferred Embodiment of the Present Invention 
The computer based system on which the currently pre- 
ferred embodiment of the present invention may be imple- 

35 mented is described with reference to FIG. 5. As noted 
above, the present invention is implemented using software 
programming instructions for execution on a computer 
based system. The computer based system and associated 
operating instructions (e.g. software) embody circuitry used 

40 to implement the present invention. Referring to FIG. 5, the 
computer based system is comprised of a plurality of com- 
ponents coupled via a bus 501. The bus 501 may consist of 
a plurahty of parallel buses (e.g. address, data and status 
buses) as well as a hierarchy of buses (e.g. a processor bus, 

45 a local bus and an I/O bus). In any event, the computer 
system is further comprised of a processor 502 for executing 
instructions provided via bus 501 from Internal memory 503 
(note that the Internal memory 503 is typically a combina- 
tion of Random Access and Read Only Memories). The 

50 processor 502 will be used to perform various operations in 
support extracting raw data from Web site, converting the 
raw data into the desired feature vectors and topology, usage 
path and text similarity matrices, categorization and spread- 
ing activation. Instructions for performing such operations 

55 are retrieved from Internal memory 503. Such operations 
that would be performed by the processor 502 would include 
the processing steps described in the flowcharts of FIG. 3 
and the accompanying descriptions. The operations would 
typically be provided in the form of coded instructions in a 

60 suitable programming language using well-known program- 
ming techniques. The processor 502 and Internal memory 
503 may be discrete components or a single integrated 
device such as an Application Specification Integrated Cir- 
cuit (ASIC) chip. 

65 Also coupled to the bus 501 are a keyboard 504 for 
entering alphanumeric input, external storage 505 for storing 
data, a cursor control device 506 for manipulating a cursor. 
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a display 507 for displaying visual output and a network 
connection 508. The keyboard 504 would typically be a 
standard QWERTY keyboard but may also be telephone like 
keypad. The external storage 505 may be fixed or removable 
magnetic or optical disk drive. The cursor control device 5 
506, e.g. a mouse or trackball, will typically have a button 
or switch associated with it to which the performance of 
certain functions can be programmed. The network connec- 
tion 508 provides means for attaching to a network, e.g. a 
Local Area Network (LAN) card or modem card with lo 
appropriate software. The network ultimately attached to is 
the Internet, but it may be through proxy servers or inter- 
mediary networks or dial-up services such as America 
' On-Line, EVodigy® or CompuServe®. 

Thus, a system for providing document change informa- 15 
tion for a community of users is disclosed. While the present 
invention is described with respect to a preferred 
embodiment, it would be apparent to one skilled in the art to 
practice the present invention with other configurations of 
digital document management systems. Such ahernate 20 
embodiments would not cause departure from the spirit and 
scope of the present invention. For example, the currently 
preferred embodiment discloses two seemingly discrete sys- 
tems having an application programming interface for sys- 
tem communication, whereas the present invention could be 25 
implemented as a single system with the document change 
agent being a part of the recommender system. 
What is claimed is: 

1. A system for providing document change information 
to a community of users, comprising: 30 

a document recommendation element for providing docu- 
ment recommendations to each user in the community 
of users based on a user profile; 

document representation storage, for storing representa- 
tions of said recommended documents; 

a document registration element coupled to said document 
recommendation element, said document registration 
element for enabhng users to identify a recommended 
document for which they would like to be notified of 
changes; 

a document change monitoring element coupled to said 
document registration element, said document change 
monitoring element for detecting a change to said 
recommended document and reporting said change, 
having a type, to said document recommendation ele- 
ment; and 

a user notification element coupled to said document 
recommendation element, said user notification ele- 
ment for notifying users in the community of users of 50 
said change when they registered with said document 
registration element to be notified of changes to said 
recommended document; 

wherein said document recommendation element further 
comprises one or more action elements for responding 55 
to said change reported by said document change 
monitoring element; each of said one or more action 
elements performing a different action related to said 
recommended document according to the type of said 
change; and 60 

wherein one of said one or more action elements takes 
action responsive to a change in content of said rec- 
ommended document, and wherein said action taken is 
to notify users of said recommended document of said 
change, to solicit feedback of said change, and to 65 
provide said feedback to other users of said recom- 
mended document in the community of users. 
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2. The system as recited in claim 1 wherein said feedback 
is made available to other users in the community of users 
not registered with said document registration element to be 
notified of changes to said recommended document upon 
their accessing the feedback. 

3. The system as recited in claim 2 wherein one of said 
one or more action elements takes action responsive to a 
change in recommended document location, and wherein 
said action is to modify location information for said rec- 
ommended document in said document representation stor- 
age. 

4. The system as recited in claim 3 wherein one of said 
one or more action elements takes action responsive to a 
determination that said recommended document has become 
unavailable; and wherein said action taken is to notify a 
systems administrator or the user who recommended said 
document. 

5. The system as recited in claim 3 wherein one of said 
one or more action elements takes action responsive to a 
determination that there exist duplicate references to said 
recommended document, and wherein said action taken is to 
modify location information for said recommended docu- 
ment in said document representation storage to reflect the 
multiple references. 

6. The system as recited in claim 3 wherein said feedback 
solicited includes one of ignoring the change and stop 
monitoring the recommended document. 

7. In a system for managing document recommendations 
to a community of users, said documents contained in a 
collection of network accessible documents, a method for 
providing document change information responsive to 
instances of change of said documents, said method com- 
prising: 

a) input from users in the community of users indicating 
to said system that they would like to be notified of a 
change to a recommended document; 

b) said system registering said recommended document 
with a monitoring agent, said monitoring agent for 
detecting changes to network accessible documents; 

c) said monitoring agent indicating to said system that a 
change to said recommended document has occurred; 

d) said system taking an appropriate action according to 
a type of change to said recommended document; 

e) said system being adapted to notify users in the 
community of users of said change when they regis- 
tered with said document registration element to be 
notified of changes to said recommended document; 

wherein said type of change to said recommended docu- 
ment is a change in content of said recommended 
document, and wherein said action taken is to notify 
users of said recommended document of said change, 
to solicit feedback of said change, and to provide said 
feedback to other users of said recommended document 
in the community of users. 

8. The method as recited in claim 7 wherein said type of 
change to said recommended document is a relocation of 
said document and said appropriate action is to change 
document representation information for said recommended 
document to reflect said relocation. 

9. The method as recited in claim 7 wherein said feedback 
solicited includes one of ignoring the change and stop 
monitoring the recommended document. 

10. The method as recited in claim 9 wherein said type of 
change to said recommended document is a determination 
that said recommended document has become unavailable; 
and wherein said action taken is to notify a systems admin- 
istrator or the user who recommended said document. 
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11. The method as recited in claim 9 wherein said type of 
change to said recommended document is a determination 
that there exist duplicate references to said recommended 
document, and wherein said action taken is to modify 
location information for said recommended document in 
said document representation storage to reflect the multiple 
references. 

12. The method as recited in claim 9 wherein said 
feedback provided by a user specifies that said recom- 
mended document is no longer of interest to the user; in the 
event no other user in the community of users is subscribed 
to be notified of changes to said recommended document, 
said system halting monitoring of changes by said monitor- 
ing agent to said recommended document. 

13. A method for obtaining evaluations of changes in 
resources accessible to a community of users of a system, 
the system including: 

resource access circuitry for accessing a set of resources; 
user notification data indicating, for each of a subset of the 

resources, one or more users in the community of users 

to be notified when a change in a resource in the subset 

of the resources is detected; and 
evaluation data indicating user evaluations of resources; 

the method comprising: 

(a) using the resource access circuitry to automatically 
attempt to access one of the resources in the subset 
and to automatically obtain access result data indi- 
cating a change to the one of the resources; 

(b) automatically applying a change notification crite- 
rion to the access result data to obtain notification 
data indicating whether to notify users in the com- 
munity of users that the resource is changed; 

(c) if the notification data indicate that users in the 
community of users should be notified that the 
resource is changed, automatically notifying users 
indicated for the resource by the user notification 
data that the resource is changed and soliciting users 
indicated for the resource by the tiser notification 
data for evaluations of the resource as changed; 

(d) receiving evaluations of the resource as changed 
from the notified users; and 
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(e) in response to the received evaluations, automati- 
cally updating the evaluation data to indicate user 
evaluations of the resource as changed and providing 
the received evaluations of the resource as changed 
^ to the users in the community of users indicated for 

the resource by the user notification data. 

14. The method of claim 13 in which the resource access 
circuitry includes a network and the resource is a network- 

20 accessible document. 

15. The method of claim 14 in which the network is the 
Internet and (a) comprises: 

providing a universal resource location (URL) for the 
resource over the network. 
^5 16. The method of claim 13 in which the access result data 
indicate whether the resource's location is changed and in 
which the change notification criterion does not require 
notification if the only change in the resource is a change in 
location. 

20 

17. The method of claim 16 in which the system further 
includes location data for the resource and in which, if the 
access result data indicate that the resource's location is 
changed, the method further comprises: 

25 updating the location data to indicate the changed loca- 
tion. 

18. The method of claim 13 in which the access result data 
indicate whether the resource's content is changed and in 
which the change notification criterion requires notification 

30 if the resource's content is changed. 

19. The method of claim 13 in which the access result data 
indicate whether the resource is available and in which the 
change notification criterion does not require notification if 
the only change in the resource is a change in availability. 

20. The method of claim 13 in which the received 
evaluations provided by a user are made available to other 
users in the community of users not indicated for the 
resource by the user notification data upon their accessing 

^„ the feedback. 

40 

* * * * * 
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